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0.  ABSTRAC 


The  Menu  and  Page  methods  of  data  entry  and  display  format  are  candidates  for 
possible  use  on  a  cathode  ray  tube  (CRT)  display  on  the  Navy  Tactical  Data  System. 
The  Menu  method  presents  all  possible  category  choices  at  a  given  level  of  data 
hierarchy;  the  operator  enters  the  number  that  corresponds  to  his  choice  of  categories 
from  the  displayed  *menu*  of  choices.  The  Page  method  displays  only  a  subset  of 
possible  choices  at  each  level  of  the  hierarchy.  If  the  desired  category  does  not 
appear,  the  operator  sequences  to  the  next  *page*  of  categories,  until  he  finds  the 
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^Reaction  time  measures  were  used  to  assess  the  relative  speed  of  data  entry  using 
six  different  types  of  problems  in  each  of  the  two  methods.  Results  showed  that 
although  each  method  took  longer  for  problem  types  requiring  deeper  hierarch ial 
structure,  there  was  no  difference  in  the  methods  themselves.  There  was  a  significant 
practice  effect,  which  was  also  independent  of  method.  Neither  method  was  preferred 
by  the  operators. 

It  is  recommended  that  the  ’page*  method  be  pursued  since  it  makes  more 
efficient  use  of  CRT  screen  area.  Further,  the  method  can  be  improved  by  organizing 
the  categories  so  that  more  frequently  occurring  items  appear  on  earlier  tpiages.* 
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FOREWORD 

This  research  and  development  was  conducted  in  response  to  a  Navy  Decision 
Coordinating  Paper,  Personnel  Supply  Systems  (NDCP-ZO 1 07-PN),  under  subproject 
Z0107-PN.08,  Information  Processing  in  Operational  Decision  Making,  and  the  sponsorship 
of  the  Chief  of  Naval  Operations  (OP-01).  The  objective  of  the  subproject  is  to  apply 
technologies  of  decision  and  information  science  in  the  development  of  management  and 
tactical  information  systems  that  are  compatible  with  human  decision-making  capabili¬ 
ties.  The  results  are  intended  for  use  by  the  Fleet  Combat  Direction  Systems  Support 
Activity  (FCDSSA),  Operation  Requirements  Section  (Code  27). 

Appreciation  is  extended  to  CDR  H.  P.  Krienke,  LCDR  W.  L.  Schwabe,  LT  A.  L. 
Maples,  and  LT  J.  L.  Dick  of  FCDSSA,  Pacific,  who  provided  the  NTDS  materials;  to  the 
Commanding  Officer  of  FCDSSAPAC  and  CDR  H.  P.  Krienke  for  allowing  FCDSSA 
personnel  to  act  as  "console  operators";  and  to  the  "operators"  themselves,  who  so 
generously  gave  of  their  time  and  provided  many  helpful  comments  in  support  of  this 
project. 


DONALD  F.  PARKER 
Commanding  Officer 

1 

h 

1 

h  k 


Problem  and  Background 

The  Navy  Tactical  Data  System  (NTDS),  a  centralized  computer-driven  information 
system,  permits  shipboard  personnel  to  both  obtain  and  enter  large  amounts  of  tactical 
information  through  a  console  containing  plan  position  indicator  and  a  direct  readout 
(DRO)  display.  Since  the  present  console's  capabilities  to  receive  and  display  information 
have  been  exceeded,  plans  have  been  made  to  replace  the  DRO  portion  of  the  console  with 
an  alphanumeric  cathode  ray  tube  (CRT)  display.  Although  this  device  will  permit 
increased  flexibility  with  respect  to  data  display  formats,  little  is  known  about  the  actual 
amounts,  types,  and  location  of  data  that  would  indeed  be  facilitating  in  the  NTDS 
environment. 

Objective 

The  purpose  of  the  present  effort  was  to  compare  two  data  entry  methods— called  the 
Page  and  Entry  methods— as  to  speed  of  data  entry  and  display.  Since  the  methods  differ 
with  respect  to  (1)  usage  of  console  data  entry  buttons,  (2)  display  format  on  the  CRT,  and 
(3)  requisite  computer  software  requirements,  a  decision  between  the  two  must  be  made 
early  in  the  design  process  (i.e.,  before  implementing  the  CRT  into  the  NTDS  system). 
Results  from  the  present  study  were  to  aid  in  making  this  decision. 

Approach 

The  two  candidate  data  entry  methods  were  simulated  on  a  Tektronix  4051  Graphics 
System.  Twelve  operators,  all  familiar  with  some  aspect  of  NTDS,  participated  in  a 
reaction  time  simulation  experiment.  The  operators  used  one  method  in  the  first  part  of 
the  experiment,  and  the  other  method,  in  the  second  part.  With  each  method,  operators 
sought  information  from  each  of  six  information  categories.  Thus,  entry  times  were 
examined  for  significance  using  a  2  (methods)  x  6  (problem  types)  x  2  (order  groups) 
analysis  of  variance. 

Findings 

1.  The  two  methods  did  not  differ  with  respect  to  speed  of  data  entry. 

2.  There  was  a  significant  practice  effect,  resulting  in  increased  speed  for  the 
second  method  tried. 

3.  The  data  entry  times  increased  as  a  function  of  the  number  of  possible  entry 
sequences. 

4.  Operators  did  not  have  a  preference  for  either  method. 

Conclusions 

Although  the  two  methods  did  not  differ  as  to  operator  preference,  the  Page  method 
makes  more  efficient  use  of  the  CRT  screen  area. 

Recommendations 

1.  Subject  to  software  constraints,  effort  should  be  directed  towards  developing  the 
Page  method. 
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2.  Alternatives  should  be  grouped  systematically  on  pages  so  that  more  frequently 
occurring  items  appear  on  earlier  pages. 

3.  To  ensure  the  evaluation  of  reasonably  organized  pages,  software  should  include 
monitors  to  determine  frequency  of  occurrence  of  alternatives  in  operational  environ¬ 
ments.  This  will  allow  reasonably  organized  pages  to  be  constructed. 

t.  Expansion  of  input  buffer  size  should  be  considered.  Although  this  would 
increase  processing  overhead,  it  would  allow  the  operator  to  complete  an  entry  task  and 
begin  another,  rather  than  have  to  time  share  between  two  (or  more)  tasks. 
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INTRODUCTION 


Background 

The  Navy  Tactical  Data  System  (NTDS),  a  centralized  computer-driven  information 
system,  permits  shipboard  personnel  to  enter  and  retrieve  large  amounts  of  tactical  data 
through  a  console.  The  console  contains  a  radar  video  plan  position  indicator  (PPI)  and  a 
number  of  fixed  format,  projection-controlled,  numeric  displays  called  the  direct  readout 
(DRO).  Since  the  variety  of  generic  types  of  data  to  be  entered  and  retrieved  expands 
constantly,  the  present  console's  capabilities  to  efficiently  receive  and  display  data  have 
been  exceeded.  At  present,  information  that  is  not  on  the  PPI  is  displayed  numerically  on 
the  DRO  display.  The  amount  of  alphanumeric  information  that  can  be  displayed, 
however,  is  severely  limited,  as  is  augmentation  of  this  capability  by  additional  labelling 
of  the  readouts.  Therefore,  the  operator  must  learn  the  meaning  of  the  numbers  in  the 
readout,  which  change,  depending  on  operating  mode  and  information  requested. 

To  alleviate  this  problem,  the  Fleet  Combat  Direction  Systems  Support  Activities, 
Atlantic  and  Pacific  Fleets,  are  engaged  in  a  program  to  supplement  and  eventually 
retrace  the  DRO  with  an  alphanumeric  cathode  ray  tube  (CRT)  Digital  Display  Indicator 
(L  J  (IP-1304/UYA-4).  The  DDI  is  far  more  versatile  than  the  DRO  because  it  can 
display  both  alphabetic  and  numeric  information.  It  provides  total  formatting  flexibility: 
The  amount,  type,  and  location  of  data  to  be  displayed  on  the  CRT  are  all  controlled  by 
the  computer,  so  that  both  data  acquisition  and  entry  by  the  operator  are  facilitated.  A 
further  advantage  is  the  DDI's  ability  to  label  (tag)  the  information.  Eventually  these 
advantages  will  be  implemented  via  computer  programming  rather  than  computer  control. 
Although  the  programming  is  subject  to  resource  constraints  imposed  by  higher  priority 
computer  tasks,  reasonable  flexibility  still  exists  for  actual  information  display. 

Problem 


The  problem  addressed  here  concerns  methods  of  entering  and  displaying  information 
on  the  DDI.  The  keyboard  of  the  present  NTDS  console  includes  18  variable  action 
buttons  (VABs),  three  rows  of  six  buttons  each.  The  functions  of  the  VABs  change  as  the 
operating  mode  of  a  console  changes  and  as  a  request  for  action  or  information  is 
processed.  As  VAB  functions  change,  so  do  the  labels,  indicating  to  the  operator  the 
current  function  of  the  VAB. 

In  principal,  it  is  a  straightforward  matter  to  indicate  the  functions  of  the  VABs  on  a 
DDI  because  of  its  programmable  flexibility.  In  practice,  however,  the  VAB  function  must 
interact  logically  with  the  operating  mode  or  method  selected  to  enter  and  display  data. 
Two  methods— called  the  Page  and  Menu  methods— have  been  selected  as  candidates  for 
implementation  into  the  NTDS.  Since  these  methods  differ  with  respect  to  (1)  usage  of 
VABs,  (2)  display  format  on  the  DDI,  and  (3)  computer  software  requirements,  a  decision 
must  be  made  between  them  early  in  the  design  process. 

Purpose 

The  purpose  of  the  present  effort  was  to  compare  the  Page  and  Menu  methods  as  to 
speed  of  data  entry  and  display  for  various  simulated  NTDS  situations.  Results  will  be 
used  in  deciding  which  method  should  be  incorporated  in  the  system. 
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APPROACH 


Data  Entry  and  Display  Methods 


The  information  representing  the  accumulated  knowledge  of  each  sensor  contact,  or 
track,  that  appears  on  the  Plan  Position  Indicator  (PP1)  lends  itself  naturally  to  an 
hierarchical  data  structure.  To  illustrate,  consider  the  informational  aspects  of  a  track 
taken  from  the  present  investigation,  which  is  presented  as  Figure  1.  The  hierarchical 
nature  of  this  information  is  manifest  in  the  natural  presentation  of  Figure  1  as  an  outline 
broken  down  under  three  levels,  indicated  by  Roman  numerals  (I,  U,  etc.),  capital  letters 
(A,  B,  etc.),  and  Arabic  numerals  (1,  2,  etc.).  Information  linked  to  the  second  level 
categorizes  the  information  under  the  first  level,  and  that  linked  to  the  third  level  further 
categorizes  the  information.  The  number  of  levels  that  could  be  included  in  such  data 
structures  is,  of  course,  arbitrary;  that  is,  Figure  1  could  be  further  broken  down  to 
include  levels  indicated  by  small  letters,  Arabic  numerals  enclosed  by  parentheses,  etc. 
The  point  is  that  the  data  are  naturally  represented  as  a  discrete,  hierarchial,  qualitative 
structure  that  can  be  easily  represented  by  computers.  The  issue  of  concern  here  is  how 
the  information  should  be  displayed  on  the  Digital  Display  Indicator  (DDI).  Although  this 
device  is  freely  formatted,  its  display  ability  is  limited  to  16  lines  and  64  columns  of 
information.  Thus,  it  is  necessary  to  determine  how  much  of  this  total  area  the  Page  and 
Menu  methods  would  use  for  information  display. 


Since  this  is  best  conveyed  by  direct  example,  the  following  paragraphs  describe  how 
the  hierarchical  data  structure  illustrated  by  Figure  1  is  displayed  by  an  operator  using 
the  two  methods.  In  each  case,  let  us  suppose  that  the  operator  is  required  to  enter 
identification  and  amplification  (ID  AMP)  information  on  a  particular  sensor  contact  or 
track,  that  this  track  is  assumed  to  be  friendly  (ASSUMD  FRND),  and  that  it  is  further 
defined  as  a  helicopter  (HELO)  (l-D-8). 


In  the  Page  method,  the  bottom  line  (of  the  16  available)  of  the  DDI  is  devoted  to 
supplying  alphabetic  labels  for  the  top  row  of  six  VABs  on  the  NTDS  keyboard.  Thus,  for 
the  above  example,  the  operator  using  the  Page  method  would  initially  see  the  display 
shown  in  Figure  2.a.  Note  that  there  are  six  labels— five  that  represent  the  first  five 
alternatives  in  the  first  hierarchial  level  of  Figure  1  and  a  sixth  marked  (PAGE).  As 
indicated  above,  each  label,  from  left  to  right,  represents  the  function  of  the  correspond¬ 
ing  button  in  the  first  row  of  VABs  on  the  console.  Since  there  are  six  alternatives  in  the 
first  level  of  Figure  1,  the  sixth,  SELECT  SEQ,  does  not  appear  in  the  initial  display.  If 
this  action  were  required  by  the  operator  using  the  Page  method,  he  would  have  to  push 
the  PAGE  button  to  make  SELECT  SEQ  available.  Each  group  of  five  (or  less,  as  in  this 
case)  labels  is  called  a  "page."  Continued  paging  will  either  present  the  next  five  (or  less) 
labels,  or,  when  the  list  of  labels  is  exhausted,  go  back  and  present  the  first  five  labels 
(page  1). 


Since,  in  the  present  example,  the  operator  wants  to  enter  identifying  and  amplifying 
information,  he  pushes  the  left-hand  button  in  Figure  2.a,  marked  ID  AMP.  This  causes 
the  first  five  alternatives  under  ID  AMP  to  appear  on  the  DDI,  as  shown  in  Figure  2.b, 
along  with  a  history  of  entries  made  so  far  (in  this  case,  only  ID  AMP).  It  also  causes  the 
functions  of  the  top  row  of  six  VABs  to  change.  Thus,  if  the  operator  were  to  push  the 
left-hand  button  at  this  point,  the  track  would  be  tagged  as  UNKNOWN.  However,  since 
he  wants  to  tag  it  as  "assumed  friendly"  (which  appears  on  the  DDI,  since  it  is  the  fourth 
alternative),  he  pushes  the  fourth  button.  This  causes  the  first  five  alternatives  under 
ASSMD  FRND  to  appear,  as  shown  in  Figure  2.c  and,  again,  a  history  of  entries  made. 
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I.  ID  AMP 

A.  UNKNOWN 

1.  cv 

2.  CG/DG 

3.  FPB 

4.  AMPHIB 

5.  NON-MIL 

B.  HOSTILE 

1.  CV 

2.  CG/DG 


6.  SURF  SUB 

7.  SIGINT 

8.  HELO 

9.  BOMBER 
10.  FIGHTER 


10.  FIGHTER 

C.  ASSMD  ENMY 

1.  CV 

2.  CG/DG 


10.  FIGHTER 

D.  ASSMD  FRND 

1.  CV 

2.  CG/DG 


10.  FIGHTER 

E.  FRIEND 

1.  CV 

2.  CG/DG 


10.  FIGHTER 
II.  MISSION 

A.  AAW 

B.  ASW 

C.  NGFS 

D.  UNREP 

E.  SAR 


F.  STRK 

G.  EW 

H.  AFW 

I.  AMPHIB 

J.  RTB 


Figure  i.  Example  of  hierarchical  data  structure. 


K.  RECCE 

L.  CAP 

M.  LOG 

N.  TANK 

O.  TRNG 
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III.  LETTER  (organized  for  Page  Method) 


A.  ABCDE 

B.  FGHI3 

C.  KLMNO 

1. 

A 

1. 

F 

1. 

K 

2. 

B 

2. 

G 

2. 

L 

3. 

C 

3. 

H 

3. 

M 

4. 

D 

4. 

I 

4. 

N 

5. 

E 

5. 

J 

5. 

O 

III.  LETTER  (organized  for  Menu  method) 

A.  A 

B.  B 

C.  C 


Z.  Z 

IV.  MISSILE  INVa 
A.  CONV 

1.  REAR  ASPT 

2.  ALL  ASPT 

3.  MULT  INT 

4.  GUIDED  A/G 

5.  UNGUID  A/G 

V.  CALLUPb 

A.  TN 

B.  EVENT  NR 

C.  SIDE  NR 

D.  SPEC  PT  NR 

VI.  SELECT  SEQ 

A.  AIR 

B.  SURFACE 

C.  SUBSURF 

D.  TNKR  ACFT 


B.  NUC 

1.  REAR  ASPT 

2.  ALL  ASPT 

3.  MULT  INT 

4.  GUIDED  A/G 

5.  UNGUID  A/G 


E.  TACAN  STA 

F.  MODE  1 

G.  MODE  2 

H.  MODE  3 


E.  INTCPTRS 

F.  HOSTILES 

G.  UNKNOWNS 

H.  ENGAGED 


I.  HELOS 
3.  ASW  ACFT 

K.  ESM  BRNGS 

L.  SONOBUOYS 


Completion  of  a  MISSILE  INV  action  required  entering  a  single-digit  number  specifying 
the  number  of  missiles. 

Completion  of  a  CALLUP  action  required  entering  a  three-digit  number  to  further 
identify  the  desired  information. 


Figure  1.  (Continued) 


ID  AMP 

:  MISSION  :  LETTER  :  MISSILE  INV  :  CALLUP 

Figure  2.a.  Initial  display. 

:  (PAGE) 

ID  AMP 

UNKNOWN 

:  HOSTILE  :  ASSMD  ENMY  :  ASSMD  FRND :  FRIEND 

Figure  2.b.  Display  after  actuating  ID  AMP  entry. 

:  (PAGE) 

ID  AMP 

ASSMD  FRND 

CV 

:  CG/DG  :  FPB  ;  AMPHIB  :  NON-MIL 

Figure  2.c.  Display  after  actuating  ASSMD  FRND  entry. 

:  (PAGE) 

ID  AMP 

ASSMD  FRND 

SURF  SUB 

:  SIGINT  :  HELO  :  BOMBER  s  FIGHTER 

Figure  2.d.  Display  after  actuating  (PAGE)  entry. 

:  (PAGE) 

Figure  2.  Page  method  displays. 
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As  indicated  above,  the  track  was  defined  as  helicopter  (HELO).  Since  HELO  is  the 
eighth  alternative  under  ASSMD  FRND,  it  does  not  appear  in  Figure  2.c.  Thus,  the 
operator  pushes  the  PAGE  button,  causing  the  requisite  label  to  appear,  as  in  Figure  2.d, 
and  the  functions  of  the  VABs  to  again  be  redefined. 

The  Page  method  has  a  number  of  advantages.  First,  reasonably  efficient  use  is  made 
of  the  screen:  only  the  bottom  three  lines  are  needed  for  VAB  definition  and  the  rest  may 
be  devoted  to  different,  independent  data  display.  Second,  the  operator  processes  a 
maximum  of  five  labels  at  any  one  time,  which  is  a  rather  light  information  load,  allowing 
for  performance  of  concurrent  tasks.  Third,  the  operator  can  page  or  enter  data  by  a 
single  button  press. 

This  method  also  has  some  disadvantages.  First,  the  operator  must  read  all  five 
labels  to  determine  whether  the  correct  one  is  present.  Second,  if  he  passes  the  correct 
page  when  paging,  he  must,  in  effect,  go  entirely  around  the  circle  before  the  correct 
page  appears  on  the  screen. 

Menu  Method 

The  Menu  method  differs  from  the  Page  method  in  two  respects.  First,  the  six  VABs 
in  the  top  row  of  the  NTDS  keyboard  are  associated  with  digits  1,  2,  3,  4,  and  5  and  the 
word  ENTER,  instead  of  directly  to  labels.  The  meanings  of  the  buttons  in  the  second  row 
are  associated  with  the  digits  6,  7,  8,  9,  and  0  and  the  word  CLEAR.  Second,  all  possible 
alternatives  are  presented,  using  as  much  of  the  screen  as  necessary. 

Both  of  these  differences  are  demonstrated  in  Figure  3.a,  which  is  the  initial  display 
seen  by  the  operator  using  the  Menu  method.  As  shown,  instead  of  label  -*•  button,  as  with 
the  Page  method,  we  have  label  -*■  digit(s)  -►  button(s).  Also,  the  sixth  alternative  in  the 
first  level,  SELECT  SEQ,  which  does  not  appear  in  the  initial  display  of  the  Page  method 
(Figure  2.a),  does  appear  in  this  display.  If  the  operator  required  this  action,  he  would 
merely  push  the  button  associated  with  the  number  of  this  action,  in  this  case,  "6”  (the 
left-hand  button  in  the  second  row),  followed  by  the  button  associated  with  the  word 
ENTER  (the  right-hand  button  in  the  first  row). 

In  the  present  example,  however,  the  operator  wants  to  enter  ID  AMP.  Thus,  he 
would  press  the  buttons  associated  with  "1"  (the  left-hand  button  in  the  first  row)  and  with 
ENTER.  This  would  cause  the  entire  "menu"  of  alternatives  under  ID  AMP  to  appear  on 
the  DDI,  as  seen  in  Figure  3.b.  At  this  point,  the  operator  would  press  the  buttons 
associated  with  "4,"  which  corresponds  to  ASMD  FRND,  and  ENTER,  causing  all  10 
alternatives  under  ASSMD  FRND  to  appear.  Thus,  the  requisite  label  is  guaranteed  to 
appear,  provided  the  entries  were  correct  up  to  that  point.  This  is  in  contrast  to  the  Page 
method,  where,  as  in  Figure  2.c,  the  requisite  HELO  does  not  appear,  and  the  operator 
must  "page"  to  find  it. 

Method  Tradeoffs 

The  processing  tradeoffs  between  the  two  methods  are  now  more  apparent.  In  the 
Menu  method,  the  operator  knows  the  correct  alternative  is  on  the  screen  somewhere;  he 
must  find  it  and  enter  the  corresponding  number.  In  the  Page  method,  the  operator  knows 
that  the  correct  alternative  may  or  may  not  appear.  If  he  finds  it  among  the  list,  he 
pushes  the  proper  button;  if  he  does  not,  he  pushes  the  PAGE  button,  and  the  process 
begins  again.  Although  the  Menu  method  does  eliminate  the  "Is  is  there?'  decision,  the 
operator  may  have  more  information  to  process. 


ID  AMP 
MISSION 


3 

4 


LETTER 
MISSILE  INV 


5 

6 


CALLUP 
SELECT  SEQ 


2  : 

3  :  4  :  5 

Figure  3.a.  Initial  display 

:  ENTER 
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2  HOSTILE 
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Figure  3.b.  Display  after  actuating"!." 
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BOMBER 

FIGHTER 


ENTER 


Figure  3.c.  Display  after  actuating 


Figure  3.  Menu  method  displays. 


In  addition,  the  actual  response  entry  differs  for  the  two  methods.  In  the  Page 
method,  each  button  is  directly  linked  to  a  label  by  virtue  of  its  relative  location.  Thus, 
the  left-hand  label  is  identified  with  the  left-hand  button,  etc.  In  the  Menu  method,  the 
buttons  are  always  linked  to  digits,  so  that  the  row  of  buttons  functions  as  part  of  a 
numeric  keyboard  entry.  The  left-hand  button  always  corresponds  to  "1,"  etc. 

Simulation 


Page  and  Menu  displays,  such  as  those  provided  by  Figures  2  and  3,  were  simulated 
using  the  BASIC  computer  programming  language  and  implemented  on  the  Center’s 
Tektronix  4051  Graphic  System.  Figure  4  is  an  example  of  the  displays  used  in  the 
experiment.  As  shown,  the  only  differences  between  such  displays  and  those  illustrated  in 
Figures  2  and  3  were: 

1.  The  action  to  be  entered  (e.g.,  ID  AMP)  appeared  in  a  rectangle  in  the  lower 
center  of  the  screen.  Each  time  an  entry  was  made,  the  information  in  this  rectangle  was 
updated. 


2.  As  digits  were  keyed,  they  appeared  on  the  screen  above  the  sixth  label.  Thus,  if 
the  operator  made  an  error  before  entering  a  digit(s),  he  could  clear  the  number  entry  by 
striking  the  CLEAR  button. 


(ENTRIES  SO  FAR) 


(LABEL  1)  :  (LABEL  2) 


ACTION  TO 
ENTER 


(digits) 


(LABEL  3)  :  (LABEL  4)  :  (LABEL  5)  :  (LABEL  6) 


Figure  4.  Example  of  display  viewed  by  subject,  including  action  to  be  entered. 


To  simulate  the  variable  action  buttons  (VABs)  on  the  NTDS  console,  a  template  was 
constructed  that  allowed  the  operation  of  a  selected  subset  (three  rows)  of  the  Tektronix 
System  keyboard.  Only  the  first  two  rows  of  VABs  were  simulated;  the  second  row  was 
exposed  but  inoperative  when  the  system  was  simulating  the  Page  method. 

The  third  row  of  the  Tektronix  keyboard  was  used  to  simulate  the  action  of  two  other 
functions  that  provided  error  recovery  capability.  First,  a  button  marked  BACKUP 
allowed  the  operator  to  return  to  the  next  higher  level  in  the  hierarchy  for  a  given 
problem.  In  effect,  this  button  allowed  him  to  erase  the  previous  entry,  unless  it  was  the 
final  entry.  He  could  not  back  up  to  a  previous  problem.  For  example,  if,  on  an  ID  AMP 
problem,  the  operator  using  the  Page  method  entered  FRIEND  instead  of  ASSMD  FRND, 
he  could  hit  BACKUP  and  the  system  would  delete  the  FRIEND  entry  and  again  present 
the  alternatives  in  that  level  of  the  hierarchy. 
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The  second  button  in  the  third  row  was  marked  ESCAPE.  If  the  operator  pushed  this 
button,  the  system  would  return  to  the  beginning  of  the  given  trial.  In  essence,  it 
performed  a  multiple  backup  function  in  the  event  (unlikely)  that  more  than  one  level  of 
entry  had  occurred  before  the  error  was  discovered. 

Finally,  under  software  control,  a  one-character  input  buffer  was  simulated.  Thus, 
operators  could  not  "stack"  keyed  inputs  except  in  the  case  of  numeric  entries,  where  the 
digits  can  be  held  in  a  temporary  store  and  sent  as  a  single  number  entry  when  the  ENTER 
button  is  pushed.  Since  the  Tektronix  system  allows  the  stacking  of  approximately  25 
characters,  the  software  algorithm  was  used  to  reject  any  intercharacter  entry  time  less 
than  100  milliseconds  as  stacked  key  strikes;  that  is,  the  second  of  the  two  characters  was 
ignored. 

Procedure 


Twelve  operators,  all  familiar  with  some  aspect  of  the  NTDS,  participated  in  the 
experiment.  These  operators  were  seated  in  front  of  the  Tektronix  4051--six  of  them 
were  given  a  set  of  instructions  on  how  to  enter  and  display  data  using  the  Page  method; 
and  six,  a  set  using  the  Menu  method  (see  Appendix  A).  They  were  then  presented  with  30 
practice  problems— five  for  each  type  of  problem  included  in  Figure  1  (i.e.,  ID  AMP, 
MISSION,  LETTER,  MISSILE  1NV,  CALLUP,  and  SELECT  SEQ)— and  were  encouraged  to 
ask  questions.  The  practice  problems  were  not  randomized  with  respect  to  type;  rather, 
all  problems  of  each  type  appeared  successively.  Thus,  there  were  five  ID  AMP  problems, 
followed  by  five  MISSION  problems,  etc.  Feedback  in  the  form  of  response  time  and 
correctness  was  given  after  each  trial.  Also,  for  a  given  problem  type,  the  30  trials 
equally  weighted  the  target  alternatives  in  groups  of  five  in  order  to  keep  the  number  of 
trials  on  each  "page"  equal.  Although  this  grouping  did  not  pertain  to  the  Menu  method,  it 
did  keep  each  target  alternative  equally  weighted. 

The  practice  session  was  followed  by  an  experimental  session,  which  consisted  of  180 
problems  (30  problems  of  each  type)  in  a  fixed,  random  order.  As  with  the  practice 
problems,  the  experimental  problems  were  the  same  and  presented  in  the  same  order  for 
both  the  Page  and  Menu  methods.  Feedback  was  given  after  each  trial,  and  2-minute  rest 
periods  were  provided  after  every  30  trials.  After  the  experimental  sessions,  the 
operators  were  asked  to  complete  a  debriefing  questionnaire  asking  them  to  give  an 
overall  rating  of  the  method  they  had  used,  to  compare  that  method  with  the  way 
analogous  entries  are  presently  being  made  in  NTDS,  and  to  provide  comments  on  its  good 
and  bad  features  and  suggestions  for  improvement  (see  Appendix  B). 

The  operators  returned  for  a  second  practice  and  experimental  session  within  a  3-day 
period.  The  procedure  was  the  same,  except  that  the  operators  were  given  instructions 
for  the  method  they  had  not  tried  in  the  first  session.  Also,  the  questionnaire  included  an 
additional  question  concerning  the  operator's  preference,  if  any,  for  either  of  the  two 
methods. 

Dependent  Measure 

The  dependent  measure  in  the  present  experiment  was  reaction  time.  Times  were 
kept  separately  for  each  of  the  six  types  of  problems,  and  for  each  stage  of  the  data  entry 
for  each  problem  type. 

For  the  Page  method,  the  response  time  recorded  was  that  which  had  elapsed 
between  the  end  of  the  screen  writing  sequence  and  the  actuation  of  one  of  the  six 
buttons  in  the  top  row.  If  multiple  "pages"  were  either  required  or  actuated,  the  times  for 
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that  level  in  the  problem  were  accumulated.  Screen-write  time  and  response  processing 
time  were  not  counted.  For  the  Menu  method,  the  response  time  recorded  was  that  which 
had  elapsed  between  the  end  of  the  screen-writing  sequence  and  the  actuation  of  the 
ENTER  button. 

Illegal  responses  were  not  recorded.  They  were  counted  in  the  total  time,  however, 
and  the  operator  was  warned  of  their  occurrence  by  an  audible  tone. 

Analysis 

The  main  dependent  measure  was  the  amount  of  time  required  to  accomplish  the 
action  entry.  An  analysis  of  variance  (ANOVA)  was  performed  using  the  average  (mean) 
total  entry  time  for  the  30  replications  of  each  type  of  problem  (i.e.,  ID  AMP,  MISSION, 
etc.)  for  each  of  the  two  methods.  Thus,  each  subject  provided  12  numbers:  an  average 
entry  time  for  each  of  six  problem  types  under  each  of  two  methods.  The  ANOVA  would 
indicate  whether  or  not  (1)  the  different  types  of  problems  were  similar  in  difficulty  level, 
or  require  varying  amounts  of  resources  (i.e.,  time)  to  accomplish,  (2)  the  display  method 
affected  data  entry  times,  and  (3)  the  order  in  which  the  methods  were  tested  affected 
the  task.  This  final  aspect  is  particularly  important  in  assessing  performance  improve¬ 
ment  due  to  practice  from  the  first  to  second  session,  independent  of  method. 


RESULTS 


The  basic  measures  of  performance  (data  entry  times)  were  recorded  separately  for 
each  trial  and  then  averaged  across  the  30  replications  to  produce  12  scores  for  each 
operator.  An  alpha  level  of  .01  was  chosen  for  test  significance. 

Analysis  of  Variance  of  Data  Entry  Times 

As  shown  in  Table  1,  two  significant  effects  emerged  from  the  analysis  of  variance 
(A VOVA)  performed  on  data  entry  times— one  for  type  of  problem  (T)  (F  =  43.3,  df  =  5,  50, 
MS  =  23.7)  and  the  other,  for  the  interaction  of  method  and  group  (M  x  G)  (F  =  43.0,  df  = 
1,  10,  MS  =  40.0).  No  other  factors  or  interactions  were  significant,  including  the  variable 
of  prime  interest,  data  entry  method  (M).  The  means  for  the  Page  and  Menu  methods 
were  4.2  and  4.3  seconds  respectively. 

Figure  5  compares  operator  performance  on  the  six  problem  types  using  the  two  data 
entry  methods.  The  function  for  each  data  entry  method  is  similar,  as  the  number  of 
possible  sequences  increases  for  each  problem  type.  Figure  5  presents  problem  types  in 
order  of  the  number  of  data  entry  sequences  required,  from  the  fewest  (SELECT  SEQ)  to 
the  most  numerous  (MISSILE  INV).  A  Tukey  multiple-paired  comparison  test  of  the 
problem  types  showed  that  SELECT  SEQ,  MISSION,  and  LETTER  differ  significantly  from 
CALLUP  and  MISSILE  INV  and  that  ID  AMP  differs  significantly  from  LETTER,  but  none 
of  the  other  comparisons  reach  significance.  Based  on  these  findings  and  the  quasi- 
categorical  nature  of  the  problem  type  factor,  no  other  inferences  can  be  made  from  this 
data. 

The  second  significant  effect  was  the  method  x  group  interaction.  This  represents  a 
practice  effect,  revealing  that  the  second  method  performed  was  the  faster,  regardless  of 
whether  it  was  the  "Menu"  or  the  "Page"  method.  This  result  was  confirmed  by  the 
summary  of  responses  to  the  debriefing  questionnaire  described  in  the  following  section. 

The  interaction  of  method  and  problem  type  (M  x  T)  approached  significance  but  fell 
short  when  conservative  degrees  of  freedom  were  applied  to  adjust  the  F  test  for  the 
violation  of  assumptions  of  independence  inherent  in  a  repeated  measures  design. 

PebriefinR  Questionnaire 

A  quantitative  summary  of  responses  to  the  debriefing  questionnaire  was  not 
attempted  since  it  was  not  designed  to  be  a  measurement  instrument.  A  visual  inspection 
of  responses,  however,  indicated  that  they  were  about  the  same  for  both  methods.  For 
example,  when  operators  were  asked  to  rate  the  two  methods  in  regard  to  speed,  ease,  and 
accuracy,  they  indicated  that  both  ranked  somewhere  between  "very"  and  "somewhat"  on 
all  three  variables.  When  asked  to  compare  the  two  methods  with  the  method  presently 
being  used  in  NTDS  consoles,  they  indicated  that  both  were  between  "better"  and  "about 
the  same"  as  to  speed,  ease,  and  accuracy. 

The  most  common  good  features  mentioned  were  economy,  speed  of  action  (i.e., 
much  could  be  accomplished  with  few  actions),  and  flexibility  in  the  sense  of  diversity  of 
data  types  that  could  be  represented.  Most  of  the  comments  regarding  bad  features 
pertained  to  the  Menu  method,  the  most  common  being  that  the  screen  contained  too 
much  information.  The  most  common  suggestion  for  improvement  concerned  the  need  to 
allow  stacked  inputs;  that  is,  to  increase  the  input  buffer  size.  The  buffer  size  limitation, 
however,  is  an  overall  NTDS  system  limitation  and  is  not  under  software  control. 
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Table  1 

Analysis  of  Variance  Summary  Table 


Source 

Error 

MS 

df 

F 

T 

(s  x  T)wg 

23.720 

5 

43.33* 

M 

<SxM)„g 

0.514 

1 

0.55 

G 

Swg 

24.100 

1 

0.90 

S 

26.670 

10 

G  x  M 

(s  X  M)wg 

40.090 

l 

43.06* 

G  x  T 

(SxTVg 

0.525 

5 

0.96 

M  x  T 

(S  x  M  x  T)wg 

0.849 

5 

4.11 

(SxM)wg 

0.931 

10 

(SxT)wg 

0.547 

50 

G  x  M  x  T 

(SxMxT)wg 

0.359 

5 

1.74 

(S  x  M  x  T)wg 

0.206 

50 

Notes. 

T  =  Problem  type  (ID  AMP,  MISSION,  etc.) 

M  =  Method  (Page  or  Menu) 

G  =  Group  (Subjects  using  Menu  method  first  and  those  using  Page  method  first). 

S  =  Subject  within  group 
wg 

*p  <  .01. 
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Figure  5.  Data  Entry  Times  as  a  Function  of  Method  and  Problem  Type. 


In  the  questionnaire  administered  after  ttie  second  session,  operators  were  asked  to 
indicate  the  method  they  most  preferred,  if  any.  Responses  to  this  question  are  shown  in 
Table  2,  which  indicates  that  operators  liad  no  clear  preference  for  either  method.  Since 
the  ANOVA  performed  on  data  entry  time  showed  that  practice  had  a  significant  effect 
on  performance,  however,  it  was  felt  that  performance  might  be  contingent  upon  which 
method  was  used  second,  particularly  if  operators  realized  they  performed  better  (e.g., 
they  were  faster)  on  the  second  method  (whichever  one  it  was),  and  thus  preferred  that 
method.  Although  data  in  Table  2  do  give  some  support  to  preference  as  a  function  of 
order,  a  Fisher  exact  probability  test  yielded  a  probability  value  of  .163,  which  is  not 
significant.  Thus,  the  lack  of  preference  for  method  appears  to  be  a  reliable  finding  and 
independent  of  the  order  in  which  the  methods  were  tried. 


Table  2 

Method  Preference  as  a  Function  of  Order 

Method  Preferred 

Group  Menu  Page  Total 

Menu  Second  5  3  8 

Page  Second  1  4  5 

Total  6  7  13 


DISCUSSION  AND  CONCLUSIONS 

Results  of  this  study  showed  that  there  is  no  real  difference  between  the  Menu  and 
Page  methods  of  data  entry.  Neither  was  convincingly  superior,  even  when  examined 
across  a  variety  of  problem  types.  Both  showed  fluctuations  between  problem  types, 
which  is,  for  the  most  part,  a  result  of  the  change  in  number  of  possible  data  entry 
sequences  as  shown  by  the  multiple,  paired  comparisons.  The  most  striking  deviation  from 
this  pattern  was  the  LETTER  problems  within  the  Menu  method.  After  completing  the 
study,  it  was  decided  that  LETTER  problems  were  not  comparable  across  the  two  methods 
because  entry  of  a  letter  followed  different  hierarchial  structures  in  the  two  methods.  In 
the  Menu  method,  only  one  level  of  choice  was  required  after  choosing  "LETTER";  in  the 
Page  method,  two  subsequent  choices  were  required.  Alleviation  of  this  difference  would 
bring  the  two  methods  more  closely  together  (Figure  5). 

Response  time  frequently  is  an  increasing  function  of  the  number  of  items  in  a 
display  (Atkinson,  Holmgren,  &  Juola,  1969,  Egeth,  Atkinson,  Gilmore,  &  Marcus,  1973; 
Teichner  &  Mocharnuk,  1974).  This  type  of  increase  recently  was  found  by  Calian, 
Curran,  and  Lane  (1977)  in  the  context  of  a  visual  search  task  using  NTDS  alphanumeric 
information  and  format.  The  best  explanation  of  the  effect  of  problem  type  in  this  study 
is  that  it  was  a  consequence  of  the  number  of  entry  sequences  and  the  number  of  items 
displayed  on  the  screen. 

The  only  other  significant  finding  was  the  practice  effect,  which  was  to  be  expected. 
Since  this  effect  was  similar  for  both  groups,  there  is  no  reason  to  suspect  that  either 
method  would  provide  more  rapid  skill  acquisition. 

Although,  while  the  two  methods  did  not  differ  with  respect  to  operator  speed  or 
preference,  the  Page  method  is  preferable  because  it  uses  display  screen  more  efficiently 
and  reduces  software  programming  time. 
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RECOMMENDATIONS 


1.  Subject  to  NTDS  software  constraints,  future  efforts  should  be  directed  toward 
developing  the  Page  method. 

2.  Alternative  choices  should  be  grouped  systematically  on  pages  so  that  more 
frequently  occurring  items  appear  on  earlier  pages. 

3.  To  ensure  the  evaluation  of  reasonably  organized  pages,  software  should  include 
monitors  to  determine  how  frequently  alternatives  occur  in  operational  environments. 

4.  Consideration  should  be  given  to  expanding  input  buffer  size.  Although  this 
would  increase  processing  overhead,  it  would  allow  the  operator  to  complete  an  entry  task 
and  begin  another,  rather  than  have  to  time  share  between  two  or  more  tasks. 
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APPENDIX  A 


INSTRUCTIONS  FOR  USING  THE  TWO  DATA  ENTRY  AND  DISPLAY  METHODS 


General  Instruction— A-l  through  A-2 

Instructions  for  Operators  Using  Menu  Method— A-3  through  A-6 
Instructions  for  Operators  Using  Page  Method— A-7  through  A- 10 
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CRT  INTERACTIVE  ENTRY 
OPERATOR  FACILITY  COMPARISON  TEST 

A.  GENERAL  INSTRUCTIONS— BOTH  METHODS 


1.0  General 

This  experiment  has  been  designed  to  test  various  methods  for  performing  entry 
and/or  display  actions  by  a  Navy  Tactical  Data  System  (NTDS)  console  operator.  You  will 
be  asked  to  complete  given  actions  in  several  different  ways.  Each  action  and  each 
method  will  be  explained  fully  and  must  be  understood  before  actually  attempting  the 
machine  entries.  You  will  be  given  two  sets  of  problems,  each  set  employing  a  different 
method.  By  analyzing  time  delays  and  button  errors  we  will  be  able  to  objectively 
identify  the  best  methods  for  implementing  standard  operator  actions.  At  the  conclusion 
of  each  series  of  runs  you  will  also  be  given  an  opportunity  to  express  your  opinion  of  the 
method  just  tested.  Recommendations  for  improvements  are  encouraged. 

2.0  Console  Description 

The  experimental  mock-up  consists  of  a  CRT  read-out  and  a  button  entry  panel.  The 
CRT  will  display  the  requested  entry  display  action,  some  accounting  data  and  the  labels 
assigned  to  six  buttons  on  the  entry  panel.  These  labels  can  be  automatically  changed  by 
the  computer  to  reflect  new  actions  assigned  to  the  buttons  as  each  run  progresses. 
Figure  1  illustrates  the  layout  of  the  CRT.  The  CRT  also  displays  the  current  selection 
made  by  the  operator  via  the  buttons. 

The  button  entry  panel  contains  14  buttons  arranged  in  two  rows  of  six  plus  two 
additional  buttons.  The  meanings  attached  to  the  top  row  of  buttons  is  indicated  by  the 
corresponding  LABEL  entries  on  the  CRT  (see  Figure  1).  These  are  computer-controlled 
action  buttons.  The  meaning  of  this  top  row  is  always  linked  to  the  CRT. 

Each  button  in  the  second  row  of  six  buttons  has  a  predefined  meaning,  as  indicated 
on  the  template  above  each  button  to  the  left  of  the  "/."  These  are  preset  action  buttons. 
In  one  special  case,  however,  they  take  on  a  different  meaning.  Whenever  the  input 
request  is  for  a  numeric  entry,  i.e.,  when  the  LABELS  on  the  CRT  are  digits,  the  second 
row  of  six  buttons  has  meaning  defined  to  the  right  of  the  "/"  on  the  template,  viz.,  the 
digitis  6,  7,  8,  9,  0,  and  "CLEAR."  As  digits  are  keyed,  they  will  appear  above  LABEL  6 
(see  Figure  1).  If  an  error  is  made  before  the  number  is  entered,  the  number  entry  can  be 
cleared  by  striking  the  "CLEAR"  button.  These  meanings  apply  only  when  the  computer 
expects  a  numeric  entry.  Use  of  these  buttons  is  considered  an  error  at  all  other  times. 

The  two  remaining  buttons  labeled  "B"  (for  Backup)  and  "E"  (for  Escape)  are  on  the 
bottom  row.  They  perform  in  the  following  manner.  In  order  to  accomplish  the  ACTION 
TO  ENTER  (Figure  1),  a  sequence  of  button  presses  must  occur.  The  "B"  and  "E"  buttons 
provide  two  ways  to  abort  the  sequence.  Pushing  the  "B"  causes  the  computer  to  backup 
one  step  in  the  sequence— it  erases  your  previous  entry.  Pushing  the  "E"  button  allows 
escape  from  the  entire  sequence  and  returns  the  system  to  the  initial  state—it  erases  all 
button  pushes  for  the  particular  problem. 


Time: 

(IN)CORRECT 


(ENTRIES  SO  FAR) 


(LABEL  1)  :  (LABEL  2) 


ACTION 
TO  ENTER 


(digits) 


(LABEL  3)  :  (LABEL  4)  :  (LABEL  5)  :  (LABEL  6) 


Figure  l 


3.0  Entering  an  Action 

The  action  you  are  required  to  enter  on  each  trial  will  appear  surrounded  by  a 
rectangle  as  in  Figure  1.  There  are  six  categories  of  action  requests  possible.  They  are: 

ID  AMP 
MISSION 
LETTER 
MISSILE  INV 
CALLUP 
SELECT  SEQ 

These  will  be  discussed  in  turn.  In  order  to  familiarize  yourself  with  the  operation  of  this 
console,  you  will  be  given  30  practice  runs  (five  of  each  of  the  above  six  in  order). 

Please  initiate  the  practice  runs  by  pushing  the  Escape  ("E")  button. 


B.  INSTRUCTIONS  FOR  OPERATORS  USING  MENU  METHOD 


3.1  First  Action:  "ID  AMP' 

The  object  is  to  associate  a  label  with  a  track  symbol  which  will  express  its  identity 
and  amplifying  data.  The  options  for  "identity"  are: 

UNKNOWN 

HOSTILE 

ASSMD  ENMY  (assumed  enemy) 

ASSMD  FRND  (assumed  friend) 

FRIEND 

The  options  for  amplifyng  data  are: 

CV  (carrier) 

CG/DG  (cruiser /destroyer) 

FPB  (fast  patrol  boat) 

AMPHIB  (amphibious) 

NON-MIL  (non-military) 

SURF  SUB  (surfaced  sub) 

SIGINT  (signal  intelligence) 

HELO 

BOMBER 

FIGHTER 

Notice  the  contents  of  the  ACTION  TO  ENTER  rectangle  ("ID  AMP  HOSTILE  FPB").  This 
is  the  information  you  are  requested  to  enter.  The  possible  choices  for  entry  at  this  stage 
(i.e.,  the  six  problem  types)  are  listed  as  a  "menu”  on  the  CRT.  The  LABEL  area  contains 
numbers,  implying  that  the  top  and  bottom  rows  of  buttons  have  a  numeric  function.  The 
choice  is  made  by  entering  the  number  corresponding  to  the  choice.  For  ID  AMP,  then, 
the  left-most  button  on  the  top  row  is  pushed.  Notice  the  appearance  of  the  digit  "1" 
above  the  ENTER  label.  The  number  is  not  given  over  to  the  computer  until  the  ENTER 
button  is  pushed. 

Push  the  "1"  button  followed  by  ENTER,  and  note  what  happens.  The  ID  AMP  appears 
in  the  ACTIONS  SO  FAR  area  of  the  CRT,  and  the  menu  changes  to  the  five  ID  options. 
Pushing  the  "2"  (HOSTILE)  followed  by  ENTER  causes  the  ACTION  SO  FAR  area  to  be 
updated,  followed  by  a  second  change  in  the  menu.  The  menu  now  contains  the  ten 
possible  amplifying  data  options.  Pushing  "3"  (FPB)  followed  by  ENTER  terminates  the 
problem.  Notice  that  the  total  (elapsed)  time  and  a  correctness  indicator  appear  in  the 
upper  left  corner  of  the  CRT.  The  time  is  long,  of  course,  since  you  are  familiarizing 
yourself  with  the  system.  The  time  should  get  shorter  as  you  get  better. 

There  is  about  a  3-second  wait  between  the  time  you  terminate  the  problem  and  the 
time  the  next  one  begins. 

Please  continue  with  the  remaining  four  ID  AMP  problems.  When  the  actions  request 
becomes  MISSION,  continue  reading  the  next  section. 

3.2  Second  Action:  "MISSION" 


The  object  is  to  associate  a  label  with  a  track  symbol  which  will  express  its  assigned 
mission.  The  options  for  "MISSION"  are: 
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AAW 

AMPHIB 

ASW 

RTB 

NGFS 

RECCE 

UNREP 

CAP 

SAR 

LOG 

STRK 

TANK 

EW 

TRNG 

AEW 

To  obtain  MISSION  options,  push  the  "2"  button  followed  by  ENTER,  and  continue  as 
before.  Continue  with  the  remaining  MISSION  problems. 

3.3  Third  Action:  "LETTER" 


H 


i 

:  . 

I 

i 

I' 

' 

l 


The  object  is  to  select  one  of  the  26  letters  for  display  at  some  undetermined 
location  on  the  radarscope. 

Press  the  "3"  button,  followed  by  ENTER,  and  notice  that  the  alphabet  appears  such 
that  a  number  is  assigned  to  each  letter  in  the  menu.  Choose  and  enter  the  appropriate 
digits,  and  then  ENTER,  as  before. 

Continue  with  the  remaining  LETTER  problems. 


3.4  Fourth  Action:  "MISSILE  INV" 

The  object  is  to  enter  a  specified  missile  type  and  quantity  as  an  inventory  associated 
with  an  undefined  air  platform.  The  options  for  weapon  type  are: 

CONV  REAR  ASPT 
CONV  ALL  ASPT 
CONV  MULT  INT 
CONV  GUIDED  A/G 
CONV  UNGUID  A/G 
NUC  REAR  ASPT 
NUC  ALL  ASPT 
NUC  MULT  INT 
NUC  GUIDED  A/G 
NUC  UNGUID  A/G 

Notice  that  in  the  ACTION  TO  ENTER  rectangle,  the  description  of  the  action  is  not 
the  order  in  which  the  action  entry  labels  appear.  Only  after  the  CONV  and  ALL  ASPT 
actions  have  been  accomplished,  can  you  enter  the  number  of  missiles.  Note  that  the 
numeric  entry  itself  does  not  require  a  menu— instead  the  reminder  ENTER  NUMBER 
appears. 


(conventional  rear  aspect) 
(conventional  all  aspect) 
(conventional  multiple  intercept) 
(conventional  guided  air-to-ground) 
(conventional  unguided  air-to-ground) 
(nuclear  rear  aspect) 

(nuclear  all  aspect) 

(nuclear  multiple  intercept) 

(nuclear  guided  air-to-ground) 
(nuclear  unguided  air-to-ground) 


Continue  with  the  remaining  four  problems. 

3.5  Fifth  Action:  "CALLUP’ 


The  object  is  to  enter  certain  minimum  identifying  data  on  a  selected  track  such  that 
the  computer  can  recognize  which  track  is  of  interest  and  present  to  the  operator  all 
known  data  on  that  track,  including  current  geographic  location.  (If  there  is  more  than 
one  track  with  the  same  minimal  identifying  data,  the  computer  would  provide  data  on  all 
in  a  sequential  manner.)  The  options  for  minimum  identifying  data  are: 


A-4 


TN  _ 

EVENT  NR  _ 

SIDE  NR  " 

SPEC  PT  NR  _ 

TACAN  STA  _ 

MODE  1 _ f 

MODE  2 _ 

MODE  3 _ 

Entering  the  numbers  here  is  the  same  as  on  Missile  Inventory  problems. 

Continue  with  the  remaining  four  problems. 

3.6  Sixth  Action:  "SELECT  SEQ" 

The  object  is  to  enter  criteria  by  which  the  computer  would  be  directed  to  call-out 
certain  types  of  tracks  in  a  sequential  manner,  all  tracks  having  a  specified  factor  in 
common.  The  options  for  the  common  factors  are; 

AIR 

SURFACE 

SUBSURF 

TNKR  ACFT 

INTCPTRS 

HOSTILES 

UNKNOWNS 

ENGAGED 

HELOS 

ASW  ACFT 

ESM  BRNGS 

SONOBUOYS 

Procedure  is  the  same  as  previously. 

4.0  Conclusion 


This  concludes  your  practice  session.  If  you  have  any  questions,  ask  them  now. 

When  the  actual  session  begins,  you  will  be  presented  with  180  problems,  30  of  each 
type.  They  will  be  in  a  mixed  order.  After  each  30  trials,  you  will  be  given  a  two  minute 
timed  rest  period.  The  session  will  take  approximately  one  hour,  including  rest  periods. 

Because  our  measure  of  the  efficiency  of  the  methods  we  are  examining  is  reaction 
time,  we  need  a  constant  time  base  for  each  method.  The  timing  clock  does  not  start 
until  the  CRT  has  concluded  its  writing  on  the  screen.  Therefore,  the  computer  is 
programmed  to  ignore  all  button  pushes  which  occur  before  the  screen  is  written.  Thus, 
although  we  wish  you  to  respond  as  rapidly  and  accurately  as  possible,  please  do  not  try  to 
get  ahead  by  pre-entering  your  response.  Wait  until  the  screen  is  written. 

Are  there  any  questions  before  we  begin? 


C.  INSTRUCTIONS  FOR  OPERATORS  USING  PAGE  METHOD 


3.1  First  Action:  "ID  AMP" 


The  object  is  to  associate  a  label  with  a  track  symbol  which  will  express  its  identity 
and  amplifying  data.  The  options  for  "identity"  are: 

UNKNOWN 

HOSTILE 

ASSMD  ENMY  (assumed  enemy) 

ASSMD  FRND  (assumed  friend) 

FRIEND 


The  options  for  amplifying  data  are: 
CV  (carrier) 

CC./DG  (cruiser /destroyer) 

FPB  (fast  patrol  boat) 

AMPHIB  (amphibious) 

NON-MIL  (non-military) 

SURF  SUB  (surfaced  sub) 

SIGINT  (signal  intelligence) 

HELO 

BOMBER 

FIGHTER 


Notice  the  contents  of  the  ACTION  TO  ENTER  rectangle  ("ID  AMP  HOSTILE  FPB"). 
This  is  the  information  you  are  requested  to  enter.  Notice  the  LABEL  area  on  the  CRT. 
The  implication  is  that  pushing  the  left-most  button  on  the  top  row  will  initiate  an  "ID 
AMP"  entry  sequence,  pushing  the  next  button  to  the  right  will  initiate  a  "MISSION"  entry 
sequence,  etc. 


Note  that  LABEL  6  on  the  CRT  is  "(PAGE)."  The  concept  here  is  that  if  there  are 
more  than  five  possible  choices  at  any  time  (there  are  six  here  because  there  are  six 
categories  of  action  request),  the  choices  are  stacked  on  "pages"  as  in  a  book.  To  examine 
the  next  page,  simply  hit  the  page  button.  Of  course,  in  the  present  problem,  this  is  not 
necessary,  since  ID  AMP  appears.  The  paging  is  circular— if  you  are  on  the  last  page  and 
you  page  again,  you  will  return  to  the  first  page. 


Hit  the  "ID  AMP'  button,  and  note  what  happens.  First,  the  button  labels  change  to 
identify  options,  and  the  "ID  AMP'  appears  in  the  ENTRIES  SO  FAR  area  (see  Figure  1)  to 
aid  you  in  keeping  track  of  where  you  are.  Hitting  the  "HOSTILE"  button  causes  the 
button  labels  to  change  to  amplifying  data  options,  and  "ID  AMP  HOSTILE"  appears  in  the 
ENTRIES  SO  FAR  area.  Pushing  the  FPB  button  terminates  the  problem.  Notice  that  the 
total  (elapsed)  time  and  a  correctness  indicator  appear  in  the  upper  left  corner  of  the 
CRT.  The  elasped  time  clock  starts  only  after  the  screen  is  written,  and  is  terminated  by 
a  button  push.  Since  the  clock  does  not  start  until  the  screen  is  written,  any  entries  made 
before  the  writing  on  the  CRT  is  complete  will  be  ignored.  You  cannot  stack  responses. 
The  time  is  long,  of  course,  since  you  are  familiarizing  yourself  with  the  system.  The 
time  should  get  shorter  as  you  get  better. 


There  is  about  a  two  second  wait  between  the  time  you  terminate  the  problem,  and 
the  time  the  next  one  begins. 
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Please  continue  with  the  remaining  lour  ID  AMP  problems.  When  the  action  request 
becomes  MISSION,  continue  reading  the  next  section. 

3.2  Second  Action:  "MISSION" 

The  object  is  to  associate  a  label  with  a  track  symbol  which  will  express  its  assigned 
mission.  The  options  for  "MISSION"  are: 


AAW 

ASW 

NGFS 

UNREP 

SAR 

STRK 

EW 

AEW 


AMPHIB 

RTB 

RECCE 

CAP 

LOG 

TANK 

TRNG 


To  obtain  mission  options,  hit  the  button  labeled  MISSION.  Notice  that  you  must  page 
once  to  get  to  STRK.  Continue  with  the  remaining  MISSION  problems. 

3.3  Third  Action:  "LETTER" 


The  object  is  to  select  one  of  the  26  letters  for  display  at  some  undetermined 
location  on  the  radarscope. 

Press  the  button  labeled  LETTER,  and  notice  that  the  alphabet  appears  such  that 
each  button  is  assigned  to  a  group  of  five  letters.  Choose  the  button  whose  letter  group 
corresponds  to  the  target  letter.  Accessing  the  "Z"  requires  paging. 

Continue  with  the  remaining  letter  problems. 

3.4  Fourth  Action:  "MISSILE  INV" 


The  object  is  to  enter  a  specified  missile  type  and  quantity  as  an  inventory  associated 
with  an  undefined  air  platform.  The  options  for  weapon  type  are: 


CONV  REAR  ASPT 
CONV  ALL  ASPT 
CONV  MULT  INT 
CONV  GUIDED  A/G 
CONV  UNGUID  A/G 
NUC  REAR  ASPT 
NUC  ALL  ASPT 
NUC  MULT  INT 
NUC  GUIDED  A/G 
NUC  UNGUID  A/G 


(conventional  rear  aspect) 
(conventional  all  aspect) 
(conventional  multiple  intercept) 
(conventional  guided  air-to-ground) 
(conventional  unguided  air-to-ground) 
(nuclear  rear  aspect) 

(nuclear  all  aspect) 

(nuclear  multiple  intercept) 

(nuclear  guided  air-to-ground) 
(nuclear  unguided  air-to-ground) 


Notice  that  in  the  ACTION  TO  ENTER  rectangle,  the  description  of  the  action  is  not 
the  order  in  which  the  action  entry  labels  appear.  After  pressing  CONV  and  ALL  ASPT, 
there  are  two  important  things  to  note.  First,  the  labels  of  the  first  five  buttons  are  now 
digits  1,  2,  3,  4,  and  5.  This  implies  that  the  second  row  of  six  buttons  is  now  active,  and 
will  represent  digits  6,  7,  8,  9,  0,  and  the  "CLEAR"  function. 
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Second,  the  sixth  button  is  now  labeled  "ENTER"  instead  of  "PAGE."  As  the  digits 
for  entry  are  pressed,  they  appear  on  the  CRT  above  the  sixth  label  (ENTER).  The 
computer  will  not  accept  anything,  however,  until  the  "ENTER"  button  is  pushed,  at  which 
point  the  entire  sequence  of  digits  is  entered  as  a  single  number.  Thus,  entering  the  "2" 
requires  pushing  the  "2"  button,  then  the  "ENTER"  button.  Hitting  "ENTER"  terminates 
the  problem. 

Continue  with  the  remaining  four  problems. 

3.5  Fifth  Action:  "CALLUP1 


The  object  is  to  enter  certain  minimum  identifying  data  on  a  selected  track  such  that 
the  computer  can  recognize  which  track  is  of  interest  and  present  to  the  operator  a  11 
known  data  on  that  track,  including  current  geographic  location.  (If  there  is  more  than 
one  track  with  the  same  minimal  identifying  data,  the  computer  would  provide  data  on  all 
in  a  sequential  manner.)  The  options  for  minimum  identifying  data  are: 

TN  _ 

EVENT  NR  _ 

SIDE  NR _ ~ 

SPEC  PT  NR  _ 

TACAN  STA _ 

MODE  1  _ 

MODE  2 _ 

MODE  3 _ 

Each  of  the  above  eight  actions  requires  a  three  digit  numeric  entry.  Entering  the 
numbers  here  is  the  same  as  on  Missile  Inventory  problems.  Note  that  paging  is  required 
to  access  IFF  Mode  entry. 

Continue  with  remaining  four  problems. 

3.6  Sixth  Action:  "SELECT  SEQ" 

The  object  is  to  enter  criteria  by  which  the  computer  would  be  directed  to  call-out 
certain  types  of  tracks  in  a  sequential  manner,  all  tracks  having  a  specified  factor  in 
common.  The  options  for  the  common  factors  are: 

AIR 

SURFACE 

SUBSURF 

TNKR  AC  FT 

INTCPTRS 

HOSTILES 

UNKNOWNS 

ENGAGED 

HELOS 

ASW  ACFT 

ESM  BRNGS 

SONOBUOYS 

Notice  here  that  paging  is  required  to  obtain  the  SELECT  SEQ  initial  action.  Subsequent 
procedure  is  the  same  as  previously. 


4.0  Conclusion 

This  concludes  your  practice  session.  If  you  have  any  questions  ask  them  now. 

When  the  actual  session  begins,  you  will  be  presented  with  ISO  problems,  30  of  each 
type.  They  will  be  in  a  mixed  order.  After  each  30  trials,  you  will  be  given  a  two  minute 
timed  rest  period.  The  session  will  take  approximately  one  hour,  including  rest  periods. 

Because  our  measure  of  the  efficiency  of  the  methods  we  are  examining  is  reaction 
time,  we  need  a  constant  time  base  for  each  method.  The  timing  clock  does  not  start 
until  the  CRT  has  concluded  its  writing  on  the  screen.  Therefore,  the  computer  is 
programmed  to  ignore  all  button  pushes  which  occur  before  the  screen  is  written.  Thus, 
although  we  wish  you  to  respond  as  rapidly  and  as  accurately  as  possible,  please  do  not  try 
to  get  ahead  by  pre-entering  your  response.  Wait  until  the  screen  is  written. 

Are  there  any  questions  before  we  begin? 
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APPENDIX  B 

DEBRIEFING  QUESTIONNAIRE 
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DEBRIEFING  QUESTIONNAIRE 


1.  Please  give  an  overall  rating  of  the  method  you  have  just  used  on  the  following  three 
aspects  by  checking  the  appropriate  box  in  each  row: 


Quick 

Easy 

Accurate 


Not 


Please  rate  this  experimental  method  compared  to  the  way  the  analogous  entries  are 
presently  made  in  NTDS.  Thus,  if  you  think  that  NTDS  as  it  presently  exists  is  on  the 
average  just  as  easy  to  use,  check  the  middle  box  in  the  second  row. 


Quick 

Easy 

Accurate 


Experimental  About  Present 

Better  the  Same  NTDS  Better 


2.  Do  you  have  any  comments  on  this  method  of  presenting  information? 
a.  Good  features  - 


b.  Bad  features  - 
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c.  Improvements 


Having  seen  this  method,  compared  with  the  existing  NTDS  entry  sequences,  can  you 
suggest  other  methods  or  sequences  of  interaction  that  would  be  easy,  quick,  and 
accurate? 


Which  method  do  you  prefer? 


Indifferent 


Note.  Item  4  included  in  questionnaire  completed  after  second  session  only, 
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